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“ Carnegie 
Overview eae 
= UL 4600 standard for AV safety cases 
e Automated driving (SAE Levels 3-5) 
e First published April 2020 
e 3 Edition published March 2023 
m= Key 4600 ideas: 
e System-level safety case provides direction 
e Vehicle as well as infrastructure and lifecycle processes all matter 
e Safety metrics used for feedback loops 
e Third party component interface protects proprietary info 
e 4600 helps you know that you've done enough work on safety 
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= Traditional safety standards are prescriptive 


e “Here is how to do safety” (process, work products) 
— ISO 26262, ISO 21448, IEC 61508, MIL-STD 882, etc. 


= UL 4600 is goal based 


e “Here is what a safety case should address” 
— Do NOT prescribe any particular engineering approach 
» Use other safety standards within the safety case context 
e Standard for how to assess a safety case 
— Minimum coverage requirement (what goes in the safety case?) 
— Properties of a well-formed safety case 
— Objective assessment criteria 
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Example 4600 Clause Mellon. 
University 

12.3.1 V&V shall provide acceptable coverage of safety related faults associated with the design phase. 
12.3.1.1 MANDATORY: 

a) Systematic design defects 

b) Design consideration of faults, corruption, data loss, and integrity loss in sensor data 

c) Requirement gaps/omissions and requirement defects 

d) Response to violation of requirement assumptions 

EXAMPLE: Response to exceptional operational environment 

e) Identification and description of the intended ODD 

f) Acceptable mitigation of aspects of the defined fault model for each component and other aspect of the item 
12.3.1.2 REQUIRED: 

a) Maintenance procedure definitions 

NOTE: While maintenance occurs during the lifecycle, the definition of procedures needs to correspond to design 

requirements and assumptions made in design regarding maintenance. 

b) Operational procedure definitions (including startup and shutdown) and operational modes 

c) Faults, corruption, data loss, and integrity loss in data from external sources 

d) Faults and failures associated with exceptional conditions that impair risk reduction functionality 

e) Hardware and software errata and other third-party component design defects 

f) Other faults in safety related functions, component designs, and other designed properties 
12.3.1.3 HIGHLY RECOMMENDED -N/A 
12.3.1.4 RECOMMENDED —N/A 
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Flexible Approaches Mellon 
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6.4.1 Each identified hazard shall be given a criticality level and assigned an initial risk assuming the 
absence of mitigation. 
6.4.1.1 MANDATORY: 
a) Hazard Log records criticality level and initial risk for each hazard 
6.4.1.2 REQUIRED: 
a) Use of at least one of the following risk evaluation approaches: 
1) Risk table 
2) Risk equation (weighted probability times severity) 
3) Fault Tree Analysis (FTA) 
4) Event Tree Analysis (ETA) 
5) Preliminary Item Safety Assessment(PSSA) 
6) Hazard Analysis and Risk Assessment (HARA) 
7) Bowtie diagram 
8) System-Theoretic Accident Model and Processes (STAMP) 
9) Field engineering feedback 
10) Other relevant risk evaluationapproaches 
b) Use of integrity level and related techniques 
EXAMPLES: Integrity level and related techniques from ISO 26262, IEC 61508; development assurance level from DO-178 


6.4.1.3 HIGHLY RECOMMENDED: 
a) Use of integrity levels defined in an accepted domain-relevant functional safety standard 
NOTE: It might not be practical to use such integrity levels for all aspects of an autonomous systems, but it is highly 
recommended to do so to the extent reasonable. 
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= Claim — a property of the system 
e “System avoids pedestrians” 

= Argument — why this is true 
e “Detect & maneuver to avoid” 

= Evidence — supports argument 
e Tests, analysis, simulations, ... 

= Sub-claims/arguments address 
complexity 
e “Detects pedestrians” // evidence 
e “Maneuvers around detected pedestrians” // evidence 
e “Stops if can’t maneuver” // evidence 


ARGUMENT 1 


EVIDENCE 1 


ARGUMENT 2 


Sub-CLAIM 2A 
~) 
Sub-ARGUMENT 2A 


EVIDENCE 2A 


Sub-CLAIM 2B 
) 
Sub-ARGUMENT 2B 


EVIDENCE 2B 
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= Everything needed to independently assess safety 
e Hazards and mitigation approaches 
e Claims traced: arguments to evidence 


m= Scope includes: Baal 
Technology: HW/SW, machine learning, tale, 

Lifecycle: deployment, operation, incidents, uinicnce: 2. 
Infrastructure: vehicle, roads, data networks, cloud computing, ... 
Road users: pedestrians, light mobility, emergency responders, ... 
Environment: Operational Design Domain (ODD) definition 

. and more ... 
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Example ODD Prompts (§8.2.2) See 


Behavioral rules Tie Ir 
e EXAMPLES: Traffic laws, vehicle path conflict resolution == ee aie 
_ E -” | . & 


priority, local customs, justifiable rule breaking for safety 


Compliance strategy of traffic rules and regulations ec fug 7? :\ 9 
e EXAMPLE: Enumeration of applicable traffic regulations and —e 7% A 2 
corresponding ego vehicle behavioral constraints https://bit.ly/2IKIZJ9 


Vulnerable populations including number, density, and types 
e EXAMPLES: Pedestrians, motorcycles, bikes, scooters, other vulnerable road users, other road users 


Special road user rules, if applicable 


e EXAMPLES: Bicycles, motorcycles, lane splitting, interacting with construction vehicles, oversize 
vehicles, snowplows, sand/salt trucks, emergency response vehicles, street sweepers, horse-drawn 
vehicles 


Seasonal effects 


e EXAMPLES: Foliage changes (e. g., leaves (dis) appearing), sun angle changes, seasonal behavioral 
patterns (e. g., summer beach traffic), seasonally-linked events (Oktoberfest, regatta crowds, fireworks 
gatherings, air shows) 
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= Safety Performance Indicator (SPI) 


e A KPI (Key Performance Indicator) specific to safety 
e Provides metrics on safety case validity 


= SPI measures: 


e Behavior metrics for safety-related behaviors 
— E.g.: Acceptable violation rate of standoff to pedestrians 
e Assumption validity within safety case 
— E.g.: Tolerates gaps of up to X meters in lane markings 
— E.g.: Correlated camera and lidar false negative rate 
e Any other metrics that validate safety case 
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SPlIs and Lifecycle Feedback ee 


= SPI: direct measurement of claim failure 
e Independent of reasoning (“claim is X ... yet here is ~X) 
e Partial measurement(s) OK; multiple SPls for a claim OK 
= A falsified safety case claim: 
e Not (necessarily) imminent loss event 
e Safety case has some defect ob) Soe 
= Root cause analysis might reveal: 
e Product or process defect 
e Invalid safety argument 
e Issue with supporting evidence 
e Assumption error, ... 


Sub-ARGUMENT 1A 
° 
EVIDENCE 1A 
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Feedback Loops Mellon 
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= Don't assume perfection — manage & improve imperfections 
e Feedback data incorporated in safety case 


SOTIF 


enti! TRIGGERING EVENTS 


ANALYSIS 


RUN-TIME 
SAFETY 
MONITOR 
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= Reused or 3" party system “component” 
e Similar in spirit to ISO 26262 SEooC 
e Hardware, software, sensor, map data, ... 


m EooC has a safety case fragment 
e Vendor need not expose that safety case 
e Instead, provides an interface containing: 
— Properties &characteristics 2H —He 
- Assumptions that system must honor _[sw»-arcument2a] 4°" Te 
— Fault model used for assessment 


- 4600 clause coverage (might be partial) 
—- Assessment report 


ARGUMENT 2 


Sub-ARGUMENT 2B 


©) 
EVIDENCE 2B 
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Complementing Other Standards ee ty 
= ISO 26262, MIL-STD 882, etc.: potential starting points 
e Still useful where applicable 


= ISO 21448 etc. for scenarios 
e Design and validation process framework 
e SaFAD and emerging standards 


= 4600 has #DidYouThinkofThat? lists 
e Initial safety case coverage 
e Learn from experience: yours; others 
e Objective assessment criteria for safety case 
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Other Key Points Mellon 
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= Version 3 explicitly addresses heavy trucks @ Evaluation of Autonomous 
=F : ‘ . Products 

= Self-certification is permitted zz 
e Internal assessor OK; no external “certificate” required is aa ee 
m Only necessary technical mitigations required Sie Iselin) 


Edition Date: March 17, 2023 


e Can use non-technical mitigations (e.g., “not applicable”) 
m Underwriters Laboratories is a non-profit SDO 

e Voting committee (STP) has diverse representation 

e Continuous Maintenance process provides timely updates 
= Does 4600 conflict with ISO 26262 or ISO 21448? 

e No 
m What if you can’t afford to buy a copy? 


e Issued standard is free to browse (“digital view”) on-line in its entirety: 


https://www.shopulstandards.com/ProductDetail.aspx?productid=UL4600 
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Review of Key Ideas Mellon 
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= System-level safety case provides direction 
e Highlights gaps in evidence and arguments 
= Vehicle, infrastructure, and lifecycle processes all matter 
e If safety case depends upon it, that makes it safety related 
= Metrics combine with feedback loops 
e Operational feedback will be essential for practical safety 
= Third party component interface to protect proprietary info 
e EooC interface permits separate component assessment 
= 4600 helps you know that you've done enough safety work 
e Robust prompts and pitfalls capture best practice/lessons learned 
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More Information 
PHILIP KOOPMAN 


= Personal UL 4600 launch page: 
e https://bit.ly/ul4600 The UL 4600 
Guidebook 


' AUTONOMOUS | 
OPERATION 


e Pointers to recorded talks 
e Pointers to viewable copies of standard 


= UL 4600 Guidebook 
e Hard copy & e-book 
e International Amazon print-on-demand 
— Country-specific info via launch page above 
e (Personal project — not officially endorsed by ULSE) 
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